Working with Metadata

About the Metadata Validator

Neurons for ITSM has a metadata validator. The metadata validator finds errors and warnings in the metadata, including mismatches and misconfigurations.

The metadata validator performs:

  • Metadata validation.

  • Database schema consistency between the database and metadata definitions.

  • Settings and data references.

  • Expression validation: Validates the expressions contained in the Neurons for ITSM database and application.

  • Group errors (only shown when Metadata is checked): Checks if there are any unused forms or lists in the application.

The tool lists any differences in related fields, missing or invalid references, and exceptions in the source and target business objects and can then fix these errors.

The metadata validator auto fix function is currently in beta mode.

Starting in Neurons for ITSM Release 2016.2, the metadata validator also finds errors and warnings for reverse relationships (also known as paired relationships) and relationships that are not well-defined, such as the following:

  • Relationships (including self relationships) that do not have a reverse relationship. This does not apply to relationships that intentionally do not have a reverse relationship or to relationships with a reverse relationship that does not have a tag that identifies the reverse pair.  

  • Relationships with cardinality but the cardinality of the reverse relationship is not the opposite. For example, if a relationship has a cardinality of 1-to-many, then the reverse relationship must have a cardinality of many-to-1 or the application displays an error. Refer to Relationship Cardinality for more information about cardinality.

  • Relationships that use a link field as a foreign key but the reverse relationship does not use the same foreign key.

  • Relationships with push constraints but the reverse relationship does not have reversed versions of the push constraints.

  • Relationships with a constraint but the reverse relationship does not have the reverse constraint.

  • Relationships where the primary key is not always the RecID.

  • Relationships where the foreign key is neither a <link>_RecID nor a validated field in the format <field>_valid.

  • Link table-based relationships with invalid cardinality. The cardinality for these relationships should always be many-to-many.

  • Relationships with a null foreign key, except for link table-based relationships.

  • Constraint-based relationships with invalid cardinality. The cardinality for these relationships should always be either one-to-one or one-to-many.

  • Constraint-based relationships with a foreign key that is not null.

  • Relationships marked with invalid direction attributes. For example, a relationship with a normal direction has the primary key in the current business object and the foreign key in the related business object and the cardinality is either one-to-one or one-to-many.

  • Relationships with invalid link field sharing across multiple business objects.

Refer to About Relationships for more information about relationships.

About Validating Metadata

The process for validating metadata is:

  1. View the metadata validation configuration and then run the validation by selecting Run Validation in the Metadata Validation Configuration workspace. Refer to Viewing the Metadata Validator and Validating Metadata.

  2. View the results of the validation run in the Metadata Validation Run workspace. Refer to Viewing Validation Results.

  3. Fix any errors that the application found, from the Metadata Auto Fix workspace. Refer to Fixing Metadata Validation Errors and Warnings.

Viewing the Metadata Validator

  1. To access the metadata validator, do one of the following:

    • In Neurons for ITSM, select More... to open the workspace selector. The list of available workspaces appears. In the Search Objects field, enter Metadata Validation Config.

    • From the Configuration Console, select Build > Automation Tools > Metadata Validation Config.

    The application displays the Metadata Validation Config workspace.

  2. Select List View to review the default validation settings. The application displays either true or false for each of the following items:

    • Metadata

    • SQL Schema

    • Settings

    • Expressions

    • Group Errors

Validating Metadata

You can only run one validation job at a time. You have to wait for the current job to finish before you start a new one.

  1. To access the metadata validator, do one of the following:

    • In Neurons for ITSM, select More... to open the workspace selector. The list of available workspaces appears. In the Search Objects field, enter Metadata Validation Config.

    • From the Configuration Console, select Build > Automation Tools > Metadata Validation Config.

    • The application displays the Metadata Validation Config workspace. By default all of the options are checked.

  2. Change the settings if needed. If you make changes, select Save. Note that changes may have been made previously

  3. Select Run Validation. The application starts the validation.

  4. While it is running, the validator lists its status. The status can be running, completed, or error. This process can take up to 30 minutes to complete, depending on the size of your tenant database.

  5. To evaluate the status of the validation, select Refresh.

Viewing Validation Results

To access the metadata validator:

  • In Neurons for ITSM, select More... to open the workspace selector. The application di splays the list of available workspaces. In the Search Objects field, enter Metadata Validation Run.

  • From the Configuration console, select Build > Automation Tools > Metadata Validation Run.

    The application displays the Metadata Validation Run workspace, including the following information:

    Field Description
    Batch Number A unique number that the application assigns every time the metadata validator runs. The application increments this number by one every time it runs.
    Database Version The current Neurons for ITSM application database metadata version.
    Status The status of the validation. Can be running, completed, or error. Only one job can have the status of running at any time.
    Progress The status of the validation if it is running.
    Metadata Displays true if the application validated the metadata, or false if it did not.
    SQL Schema Displays true if the application validated the Microsoft SQL schema, or false if it did not.
    Settings Displays true if the application validated the settings, or false if it did not.
    Expressions Displays true if the application validated the expressions, or false if it did not.
    Group Errors Displays true if the application validated the group errors, or false if it did not.
    Started By The name of the user who ran the validation.
    Started On The date and time when the validation started.
    Ended On The date and time when the validation completed.
    Fix Status The status of the auto fix feature. Can be either running or not running.
    Fix Run Date Time The date and time when the auto fix feature was last run.

Viewing Specific Validation Results

  1. Access the metadata validator as described in Viewing Validation Results.

  2. To review the results of a specific job, double-click the job number. The application displays the following information on each of the following tabs:

    Field Description
    Batch Mode Enum

    The type of validation. Can be one of the following:

    Expressions

    Metadata

    Settings

    SQL Schema
    Metadata Type Enum The business object type, such as layout, quick action, rule, rule override, table, and so on.
    Metadata Sub Type Enum The subcategory or the key items inside the metadata such as the table metadata, calculated rules, fields, indexes, form, list, data references, and so on.
    Table Ref The name of the table with the message.
    Message Type Enum The message type, such as expression parse error, field not found in relationship, and so on.
    Message Severity

    The severity of the message. Can be one of the following:

    Error

    Warning

    Information

    Validation
    Message A text message with detailed information.

  3. To review the errors, select the Errors tab. The application displays the errors that apply to this validation run. You can fix certain errors by using the metadata autofix feature. Refer to Fixing Metadata Validation Errors and Warnings. The following table lists the errors that you can fix, but does not list the errors that cannot be fixed.

    Message Type Meaning
    ChildPanelUsesMissedRelationship

    The child tab in the layout references a relationship that no longer exists. The validator code allows usage of specific group member business objects (for example, CI.Computer) as long as the relationship references the corresponding group base business object (for example, CI). The auto fix removes from the layout definition any erroneous child tabs that do not have a valid relationship assigned to them.
    FieldDiffersFromBaseTable

    The attributes for a field in a group member business object (for example, CI.Computer) must not differ from those of the group base business object (for example, CI). The auto fix reconciles the properties of the base and member business object fields such as data type and length, so that all the fields have the accommodating property. For example, if a base business object text field has a size of 50 characters and is defined as text, but the member business object field has a size of 100 characters and is defined as Unicode text, the fields in the base and all member business objects are updated to be 100 characters and Unicode text.
    FieldDiffersFromContractTable

    The attributes for a field in a concrete business object must be the same as the attributes of the same field in any contract that it implements. The auto fix reconciles the properties of the contract and concrete business object fields such as data type and length, so that all the fields have the accommodating property. For example, if a contract business object text field has a size of 50 characters and is defined as text, but the concrete business object field has a size of 100 characters and is defined as Unicode text, the fields in the contract and all concrete business objects are updated to be 100 characters and Unicode text.

    FieldNullableMismatch

    The nullable attribute of a business object field does not match the actual nullable state of the corresponding column in the table or view. The auto fix tries to set the Allow Null property of the column to what is set in the field. If the field is set to false and the database column has a null value, the auto fix logs an exception and the auto fix fails.

    FieldShouldBeUniqueInRelationship

    To enforce the relationship constraint for one-to-one relationships, the foreign key field must be marked as unique. This fix marks the foreign key field as unique by creating a unique index. The auto fix does not precheck for redundant data in the foreign key field, so a runtime error with the unique key error will result during the auto fix in such cases. At this point, you must either manually update the relationship so that it is no longer a one-to-one relationship, or remove the redundant data in the auto fix column.

    IndexNotFoundForUniqueField

    A field that is marked as unique must have a corresponding metadata unique index. To check this, the validator does not match based on naming conventions (such as "_Unique" suffix), but looks in the database for such an indexes, before reporting an error. If there is no such index, this fix creates one in both the metadata and the database.

    MissingExpressionTreeError

    For metadata consistency, all expression references must refer to a valid expression and not be null. The validator identifies cases of null expressions. This fix creates an expression that returns a constant null to replace each null references.
    PairedRelationshipCardinalityDiffer

    Relationships with cardinality but the cardinality of the reverse relationship is not the opposite. For example, if a relationship has a cardinality of 1-to-many, then the reverse relationship must have a cardinality of many-to-1 or the application displays an error.
    PairedRelationshipRelatedForeignKeyDiffer Relationships that use a link field as a foreign key but the reverse relationship does not use the same foreign key.
    PairedRelationshipSameDirection Relationships marked with invalid direction attributes. For example, a relationship with a normal direction has the primary key in the current business object and the foreign key in the "other" business object and the cardinality is either one-to-one or one-to- many.

    RedundantRecIdForConcreteObject

    		 There are concrete business objects (for example, Account) that inherit their RecID from a contract that it implements (for example, Frs_CompositeContract_Entity). The field must not be inherited, so that only the RecID field in the concrete business object is present. The auto fix removes the redundant RecID field which is inherited from the corresponding contract business object, leaving only the RecID column directly from the concrete business object as the sole RecID field.

    SqlSchemaColumnDataTypeIncompatible

    The validator checks for incompatible data types for each field in a business object and the corresponding column in the database table or view. The auto fix tries to update the column data type in the database and corresponding changes in metadata. The auto fix does not update cases like, if CPUCount is defined as numeric in a business object CI#Computer as compared to varchar in database table.

    SqlSchemaColumnLengthMismatch

    The validator checks for length mismatch for a business object field as described by the metadata and the corresponding column in the database table or view. The auto fix expands either the field length or column length to match the other length. For example, if the field length is 10 and the column length is 20, the auto fix updates the field length to 20. Alternatively, if the field length is 20 and the column length is 10, the auto fix updates the column length to 20.

    SqlSchemaIndexNotFoundForUniqueField

    		 A field that is marked as unique does not have a corresponding unique index in the database. The auto fix creates a new unique index for the field.

    SqlSchemaPrimaryKeyNotFound

    		 There is no primary key index for the database table or view based on the primary key field designated in the business object (typically the RecID). The auto fix creates a new primary key index for the key field.

  4. To review the warnings, select the Warnings tab. The application displays the warnings that apply to this validation run. You can fix certain warnings by using the metadata autofix feature. Refer to Fixing Metadata Validation Errors and Warnings. The following table lists the warnings that you can fix, but does not list the warnings that cannot be fixed.

    Message Type Meaning

    FieldNullableMismatch

    		 The nullable attribute of a business object field does not match the actual nullable state of the corresponding column in the table or view. The auto fix tries to set the Allow Null property of the column to what is set in the field. If the field is set to false and the database column has a null value, the auto fix logs an exception and the auto fix fails.

    RelatedFieldLengthMismatch

    		 A validated field in a business object has a different field length than the key field of the corresponding validation business object. The auto fix updates the validated field to have the same size as the field from the corresponding business object. For example, assume that the Status field in the Incident business object is defined with a length of 50, and is validated from the Status field in the IncidentStatus business object. If the latter field is defined with a length of 100, then the auto fix updates the Status field in the Incident business object to be Unicode text to be 100 as well.

    RelatedFieldTypeMismatch

    		 A validated field in a business object has a different field type than the key field of the corresponding validation business object. The auto fix updates the validated field to have the same type as the field from the corresponding business object. For example, assume that the Status field in the Incident business object is defined as text, and is validated from the Status field in the IncidentStatus business object. If the latter field is defined as Unicode text, then the auto fix updates the update the Status field in the Incident business object to be Unicode text as well.

    SqlSchemaColumnDataTypeIncompatible

    The validator checks for incompatible data types for each field in a business object and the corresponding column in the database table or view. The auto fix tries to update the column data type in the database and corresponding changes in metadata. The auto fix does not update cases like, if CPUCount is defined as numeric in a business object CI#Computer as compared to varchar in database table.

    SqlSchemaIndexNotFound

    		 The validator checks for an index defined for the Microsoft SQL table or view, but does not exist in the corresponding metadata definition. The auto fix creates an index in the metadata based on the index in the database.

    SqlSchemaIndexNotFoundForUniqueField

    		 A field that is marked as unique does not have a corresponding unique index in the database. The auto fix creates a new unique index for the field.

    SqlSchemaPrimaryKeyNotFound

    There is no primary key index for the database table or view based on the primary key field designated in the business object (typically the RecID). The auto fix creates a new primary key index for the key field.

    UnusedForm

    		 The validator checks for forms that are not referenced by any layout definitions. This fix deletes all such forms.

    UnusedGrid

    		 The validator checks for lists that are not referenced by any layout definitions. This fix deletes all such lists.
    UnusedPickList There are pick lists that are not referenced by any layout definitions.

  5. To review the validation messages, select the Validations tab. The application displays the validation messages that apply to this validation run.

  6. To review all of the errors, warnings, and validation messages, select the All tab. The application displays all of the errors, warnings, and validation messages that apply to this validation run.

  7. To view the validation configuration details, select the Validation Configuration tab.

Fixing Metadata Validation Errors and Warnings

After you have configured and run the metadata validator, the application populates the Metadata Auto Fix (BETA) workspace with information about available fixes. The available fixes are grouped according to category and the application lists the last date that the autofix feature was run for each category. Each category has its own Run button, so you must fix each one individually.

We recommend that you validate the metadata just before running the autofix feature. Check the value of the Ended On field in the Metadata Validation Config workspace to review the last date when the autofix feature was run.

Metadata Validation Auto Fix Categories

Expressions

  • Missing Expression Tree

  • Simplify Expressions

Metadata

  • Child Panel Uses Missed Relationship

  • Field Differs From Base Table

  • Field Differs From Contract Table

  • Field Should Be Unique In Relationship

  • Index Not Found For Unique Field

  • Redundant RecId For Concrete Object

  • Related Field Length Mismatch

  • Related Field Type Mismatch

  • Unused Form

  • Unused Grid

Database

  • DB Column Data Type Mismatch

  • DB Full Text Index Not Found

  • DB Index Not Found

  • DB Index Not Found For Unique Field

  • Incompatible DB Column Data Type

  • Index Not Found in Metadata

  • Mismatch in DB Column Length

  • Mismatch in DB Field Nullable Attribute

  • Missing DB Table Primary Key

Using the Auto Fix Feature

To fix the errors, you must run the metadata auto fix for one error at a time. After fixing an error, you must validate the data again before fixing another error because fixing one error might change the data.

Before you begin, validate the data. Refer to Validating Metadata.

  1. From the Configuration console, select Build > Automation Tools > Metadata Auto Fix (BETA).

  2. Review the list of available fixes.

  3. Select Run next to a category to fix the error. The application displays Started under the Run button.

  4. Select Reload Auto Fixes to reload the page. The application refreshes the page and removes any errors that have been fixed. The application updates the value of the Last run field.

  5. To fix another error, revalidate the data (refer to Validating Metadata), and repeat steps 1-4.